membacking: add private anonymous memory backend for guest RAM#2849
membacking: add private anonymous memory backend for guest RAM#2849jstarks merged 11 commits intomicrosoft:mainfrom
Conversation
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
Add commit() and decommit() methods to SparseMapping for managing the commit state of pages within an existing VA reservation. On Windows: - decommit() calls VirtualFreeEx with MEM_DECOMMIT to release physical pages while keeping the VA range reserved - commit() calls VirtualAlloc2 with MEM_COMMIT to make pages accessible, working idempotently on already-committed pages On Linux: - decommit() calls madvise(MADV_DONTNEED) to release pages back to the kernel (subsequent reads return zeroes) - commit() is a no-op since the kernel handles page faults transparently for MAP_ANONYMOUS regions These primitives enable a private memory mode where guest RAM uses anonymous virtual memory instead of file-backed shared memory sections.
Add a private_ram flag to VaMapper that enables a mode where the backing SparseMapping uses committed anonymous memory instead of file-backed mappings for guest RAM. In private-RAM mode: - page_fault() on Windows commits 64KB-aligned chunks via SparseMapping::commit() and returns PageFaultAction::Retry, allowing the hypervisor to retry the faulting access - page_fault() on Linux returns PageFaultAction::Fail since the kernel handles faults transparently for MAP_ANONYMOUS regions - alloc_range() eagerly commits a range of pages - decommit() releases pages back (for future balloon/free-page-reporting) Add new_private_mapper() to MappingManagerClient that creates a VaMapper in private-RAM mode, bypassing the mapper cache since private mappers cannot be shared across partitions.
Add a private_memory option to GuestMemoryBuilder that allocates guest RAM from committed anonymous virtual memory instead of a shared memory section. When private_memory is enabled: - No shared memory file/section is created (guest_ram is None) - A private VaMapper is used via new_private_mapper() - RAM ranges are eagerly committed with alloc_range() - File-backed add_mapping() calls are skipped for RAM regions - Memory prefetch is disabled (pages are already committed) Validation rejects private_memory combined with: - Legacy memory layout (x86_legacy_support) which requires shared memory - Pre-existing memory backing (shared_memory_region_base) The guest_ram field is changed to Option<Mappable> to reflect that private memory mode does not produce a shareable backing object.
Add the --private-memory CLI flag to OpenVMM and wire it through the configuration pipeline: - Add private_memory field to MemoryConfig in openvmm_defs - Add --private-memory boolean flag to CLI argument parser - Wire the flag from CLI args to MemoryConfig in openvmm_entry - Pass private_memory to GuestMemoryBuilder in dispatch.rs - Default private_memory to false in ttrpc and petri config construction
Add unit tests covering the new private memory functionality: sparse_mmap (3 tests): - test_decommit_zeros_pages: verify decommit returns pages to zero - test_commit_after_decommit: verify commit restores access after decommit - test_commit_idempotent: verify commit on already-committed pages is safe membacking (4 tests): - test_private_ram_alloc_write_read: basic alloc/write/read cycle - test_private_ram_decommit_zeros: verify decommit zeros private pages - test_private_ram_recommit_after_decommit: verify recommit after decommit - test_private_ram_commit_idempotent: verify double-commit is safe All tests use direct SparseMapping operations to avoid async infrastructure complexity, exercising the actual platform commit/decommit code paths.
There was a problem hiding this comment.
Pull request overview
This PR adds a --private-memory mode to allocate guest RAM as private anonymous memory (instead of shared section/file-backed mappings), enabling future confidential VM scenarios where host aliasing of guest-private pages is disallowed.
Changes:
- Add
commit()/decommit()support tosparse_mmap(VirtualAlloc/VirutalFree on Windows;madvise(MADV_DONTNEED)/no-op commit on Unix). - Add private-RAM mode to
membacking’sVaMapper/GuestMemoryBuilder, plus config/CLI plumbing (--private-memory) and a Petri boot test. - Wire the new config through OpenVMM entrypoints (CLI + ttrpc) and Petri config construction.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| vmm_tests/vmm_tests/tests/tests/multiarch.rs | Adds an end-to-end boot test for --private-memory. |
| support/sparse_mmap/src/windows.rs | Implements Windows commit()/decommit() for sparse mappings. |
| support/sparse_mmap/src/unix.rs | Implements Unix decommit() via madvise(MADV_DONTNEED) and no-op commit(). |
| support/sparse_mmap/src/lib.rs | Adds unit tests covering commit/decommit behavior. |
| petri/src/vm/openvmm/construct.rs | Extends OpenVMM config construction to include new memory config fields. |
| openvmm/openvmm_entry/src/ttrpc/mod.rs | Plumbs new memory config fields into ttrpc-created configs. |
| openvmm/openvmm_entry/src/lib.rs | Wires --private-memory CLI option into MemoryConfig. |
| openvmm/openvmm_entry/src/cli_args.rs | Adds --private-memory flag. |
| openvmm/openvmm_defs/src/config.rs | Extends MemoryConfig with private_memory (and pcie_ecam_base). |
| openvmm/openvmm_core/src/worker/dispatch.rs | Passes private_memory into GuestMemoryBuilder. |
| openvmm/membacking/src/memory_manager/mod.rs | Implements private-memory build path and updates RAM backing handling. |
| openvmm/membacking/src/mapping_manager/va_mapper.rs | Adds private-RAM behavior (alloc/commit/decommit) + tests. |
| openvmm/membacking/src/mapping_manager/manager.rs | Adds new_private_mapper() using the existing mapper cache. |
You can also share your feedback on Copilot code review. Take the survey.
The pcie_ecam_base field and DEFAULT_PCIE_ECAM_BASE constant were removed from main but got reintroduced during a rebase. Remove the field from MemoryConfig and all three usage sites.
- shared_memory_backing() now returns Option<SharedMemoryBacking> instead of panicking in private memory mode - Restart path returns a proper error when private memory is active instead of hitting an expect() abort - VaMapper private-RAM fault paths use GuestMemoryErrorKind::Other instead of OutOfRange, since commit failures may be resource-related rather than address-range issues
Use std::io::Error::other() instead of std::io::Error::new(ErrorKind::Other, ...) as required by the io_other_error clippy lint.
- Include MemoryRange in PrivateRamAlloc error for better diagnostics - Move private_ram flag into MappingManagerClient, initialized at MappingManager::new() time, instead of using a separate new_private_mapper() method with implicit caching
smalis-msft
left a comment
There was a problem hiding this comment.
Chris really should look at this, but I think it's ok?
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.
You can also share your feedback on Copilot code review. Take the survey.
Remote mappers commit anonymous pages in the remote process's address space, making them inaccessible locally. Return VaMapperError::RemoteWithPrivateMemory instead of creating a broken mapper.
Summary
Add a
--private-memorymode that allocates guest RAM using private anonymousmemory (VirtualAlloc on Windows, mmap MAP_ANONYMOUS on Linux) instead of
page-file-backed shared memory sections. This is the foundation for supporting
confidential VM scenarios where guest memory must not be aliased or backed by a
named file mapping.
Motivation
Today, OpenVMM allocates guest RAM as a shared memory section (file mapping)
that gets mapped into the VMM process. This section-backed model makes it
easy to share RAM between processes, but it's fundamentally incompatible with
hardware-enforced private memory (SEV-SNP, TDX, Hyper-V VSM): once a page is
assigned to a guest as private, the host can't map it. For those scenarios we
need a memory model where the VMM manages virtual address ranges directly
using commit/decommit, without a backing file object.
Approach
Layer 1:
sparse_mmap—commit()/decommit()VirtualAlloc2(MEM_COMMIT)/VirtualFreeEx(MEM_DECOMMIT)madvise(MADV_DONTNEED)Layer 2:
VaMapper— private-RAM modeprivate_ramflag onVaMapper; when set,page_fault()commits memoryon demand (Windows) or returns
Failsince Linux auto-faults.alloc_range()eagerly commits a VA range;decommit()releases it.new_private_mapper()onMappingManagerClientcreates a private mapper usingthe existing
MAPPER_CACHEfor correct single-instancing.Layer 3:
GuestMemoryBuilder— conditional allocation.private_memory(true)builder option.alloc_shared_memory()for guest RAM; uses the private mapper instead.alloc_range()during build.add_mapping()for RAM ranges (no file-backed mapping to register).Layer 4: CLI / config wiring
--private-memoryflag incli_args.rs, plumbed throughMemoryConfigtothe builder.
Layer 5: Petri boot test
boot_private_memorytest inmultiarch.rsvalidates end-to-end boot withprivate memory enabled.
Testing
sparse_mmapunit tests + 5membackingunit tests, all passing onboth Linux and Windows (
x86_64-pc-windows-msvccross-compile)--private-memorycargo xtask fmt --fixandcargo doccleanFuture work
decommit()to the balloon driver so deflatedpages release physical memory (Phase 4 from the design).